Linear-quadratic control and information relaxations

We apply the recently developed duality methods based on information relaxations to the classic linear quadratic (LQ) control problem. We derive two dual optimal penalties for the LQ problem when the control space is unconstrained. These two penalties, which are derived using value function and gradient methods, respectively, may be used to evaluate sub-optimal policies for constrained LQ problems when it is not possible to determine the optimal policy exactly. We also compare these dual penalties to the dual penalty of Davis and Zervos (1994). This connection to the earlier work of Davis and Zervos is not widely known and demonstrates that some of these duality ideas have been in circulation for some time. We also emphasize that while the three penalties are dual optimal, they are not identical. Indeed their differences have significant implications when the penalties are used via Monte-Carlo to evaluate sub-optimal policies for constrained LQ problems. Our conclusions should apply more generally to other stochastic control problems.